Skip to content

feat(planner): record observed facts and unknowns in the cold AIS identity - #17743

Draft
tedzhouhk wants to merge 1 commit into
mainfrom
hzhou/planner-ais-fitted-identity
Draft

tedzhouhk wants to merge 1 commit into
mainfrom
hzhou/planner-ais-fitted-identity

Conversation

@tedzhouhk

@tedzhouhk tedzhouhk commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Overview

This PR is a draft until an AISimulate nightly carries the stack ai-dynamo/aisim#669 → ai-dynamo/aisim#670 → ai-dynamo/aisim#671 → ai-dynamo/aisim#658. Once that nightly exists, this PR will bump the AISimulate pin and become ready.

Without ais_perf_model, the Planner builds a cold fpm_regression model. Its identity used system: "unprofiled" and parallel widths of 1, whatever the deployment. A state snapshot saved from that model therefore claimed facts the Planner never observed, and AISimulate could not use it to time a recommend / predict role.

The AISimulate stack adds three things:

  • regression and learned replay;
  • "unknown" identity facts and a deployment_limits block;
  • recommendation roles timed by such snapshots.

This PR makes the Planner record its cold identity in that format.

Key Changes

components/src/dynamo/planner/core/perf_model/ais_adapter.py, _build_ais_config (cold branch only; a configured ais_perf_model is unchanged):

  • Recorded from what the worker reports:
    • model: model_name, or "unknown";
    • backend;
    • kv_block_size and nextn, as before;
    • fallback_policy: deny, now explicit;
    • deployment_limits, from EngineCapabilities: num_gpus, max_num_batched_tokens, max_num_seqs, max_kv_tokens, context_length. A missing or non-positive value becomes "unknown".
  • Recorded as "unknown":
    • system and backend_version;
    • the quantization modes and attention_backend;
    • tp, pp, attention_dp, moe_tp_size and moe_ep_size, for a worker with more than one GPU. A one-GPU worker records every width as 1.
  • Legacy-options migration: the migration that supplies estimator defaults keeps native placeholders, because only its estimator_config is used.
  • Docs: the planner guide, in English and zh-CN, describes the recorded identity.

Validation

All runs used python3 -m pytest, with PYTHONPATH set to this worktree's components/src and to the AISimulate stack head (ai-dynamo/aisimulate#658 at 9f233acf):

Tests Result
components/src/dynamo/planner/tests/unit/test_ais_perf_config.py 27 passed
components/src/dynamo/planner/tests/unit 1468 passed

The new tests check two things:

  • What a one-GPU worker with full capabilities records, and that AIS constructs the cold model.
  • What a wider worker with partial capabilities and no model name records.

pre-commit run --files <changed files> and docs/fern/scripts/docs_lint.py pass.

This PR is not validated against the currently pinned AISimulate. It needs the nightly above, which is why it is a draft.

Issues and Ongoing Work

Pull Request Checklist
  • I used Dynamo's /pr-checklist skill.

🤖 Generated with Claude Code

…ntity

Without ais_perf_model, the Planner built its cold fpm_regression model
with system "unprofiled" and default parallel widths of 1, whatever the
deployment. A state snapshot saved from it therefore claimed facts the
Planner never observed, and AISimulate could not use it to time a
recommendation role.

The cold identity now records what the worker reports (model, backend,
KV block size, MTP depth) and, as deployment_limits, the GPUs per
worker, scheduler limits, KV capacity and context length. Facts the
Planner cannot observe are AISimulate's "unknown": hardware, backend
version, quantization, attention backend, and the parallel split of a
worker with more than one GPU (one GPU fixes every width to 1). The
legacy-options migration that supplies estimator defaults keeps native
placeholders, since only its estimator controls are used.

Requires an AISimulate nightly that accepts "unknown" identity facts and
deployment_limits (ai-dynamo/aisim#658 and its stack).

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added feat documentation Improvements or additions to documentation planner labels Oct 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@devin-ai-integration

Copy link
Copy Markdown
Contributor

❌ Dynamo PR CI failed — run 41153898441 (attempt 1) on 39f0a5dff4

Gate checks: ❌ backend-status-check · ✅ deploy-status-check · ❌ dynamo-status-check

Other Jobs
dynamo-runtime ❌ 4 ✅ 3
planner ❌ 2 ✅ 3

Failure details

6 jobs failed, all from one error: ForwardPassPerfModelConfig.__init__() rejects the deployment_limits keyword that dynamo/planner/core/perf_model/ais_adapter.py:300 (_build_ais_config) passes to it. Planner unit/integration tests fail directly, and the replay aisimulate CLI e2e tests fail in a subprocess. There are no infrastructure signatures and no cancelled jobs.

❌ planner / CPU Test cpu, arm64: TypeError: unexpected keyword argument 'deployment_limits'

Job: planner / CPU Test cpu, arm64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 124398266810 -R ai-dynamo/dynamo --log-failed

components/src/dynamo/planner/core/state_machine.py:110: in __init__
    self._prefill_regression = PlannerEnginePerfModel(
components/src/dynamo/planner/core/perf_model/ais_adapter.py:137: in __init__
    self._init_engine_model()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:154: in _init_engine_model
    model_key = self._model_key()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:397: in _model_key
    json.dumps(self._build_ais_config(), sort_keys=True, separators=(",", ":")),
components/src/dynamo/planner/core/perf_model/ais_adapter.py:300: in _build_ais_config
    ForwardPassPerfModelConfig(
E   TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
=========== 82 failed, 2747 passed, 24 skipped in 635.62s (0:10:35) ============

82 tests fail with the same TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits', raised from PlannerEnginePerfModel._build_ais_config (ais_adapter.py:300) whenever a planner perf model is built. Affected files: test_engine_adapter.py (44), test_reconfiguration.py (18), test_ais_perf_config.py (5), test_signal_health.py (3), test_pending_startup_scaling.py, test_retune_engine_adapter.py, test_simulation_integration.py (2 each), plus single tests in test_ais_native_estimator.py, test_replay_adapter_fpm.py and test_replay_retune.py; the other 3 failures are the test_unified_cli_dynamo.py aisimulate CLI cases, which hit the same error in a subprocess.

❌ planner / CPU Test cpu, amd64: TypeError: unexpected keyword argument 'deployment_limits'

Job: planner / CPU Test cpu, amd64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 124398266854 -R ai-dynamo/dynamo --log-failed

components/src/dynamo/planner/core/state_machine.py:110: in __init__
    self._prefill_regression = PlannerEnginePerfModel(
components/src/dynamo/planner/core/perf_model/ais_adapter.py:137: in __init__
    self._init_engine_model()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:154: in _init_engine_model
    model_key = self._model_key()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:397: in _model_key
    json.dumps(self._build_ais_config(), sort_keys=True, separators=(",", ":")),
components/src/dynamo/planner/core/perf_model/ais_adapter.py:300: in _build_ais_config
    ForwardPassPerfModelConfig(
E   TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
=========== 82 failed, 2747 passed, 24 skipped in 490.20s (0:08:10) ============

The same 82 failures as the arm64 job, all from the same deployment_limits TypeError in ais_adapter.py:300.

❌ dynamo-runtime / test / sequential cuda13.0, amd64: TypeError: unexpected keyword argument 'deployment_limits'

Job: dynamo-runtime / test / sequential cuda13.0, amd64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 124399138051 -R ai-dynamo/dynamo --log-failed

components/src/dynamo/planner/core/state_machine.py:110: in __init__
    self._prefill_regression = PlannerEnginePerfModel(
components/src/dynamo/planner/core/perf_model/ais_adapter.py:137: in __init__
    self._init_engine_model()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:154: in _init_engine_model
    model_key = self._model_key()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:397: in _model_key
    json.dumps(self._build_ais_config(), sort_keys=True, separators=(",", ":")),
components/src/dynamo/planner/core/perf_model/ais_adapter.py:300: in _build_ais_config
    ForwardPassPerfModelConfig(
E   TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
== 80 failed, 6287 passed, 158 skipped, 2814 deselected in 1545.09s (0:25:45) ==

80 planner/replay tests fail with the same TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits' from ais_adapter.py:300: the planner test files listed above, plus components/src/dynamo/replay/tests/test_retune_replay.py (1).

❌ dynamo-runtime / test / sequential cuda13.0, arm64: TypeError: unexpected keyword argument 'deployment_limits'

Job: dynamo-runtime / test / sequential cuda13.0, arm64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 124399138078 -R ai-dynamo/dynamo --log-failed

components/src/dynamo/planner/core/state_machine.py:110: in __init__
    self._prefill_regression = PlannerEnginePerfModel(
components/src/dynamo/planner/core/perf_model/ais_adapter.py:137: in __init__
    self._init_engine_model()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:154: in _init_engine_model
    model_key = self._model_key()
components/src/dynamo/planner/core/perf_model/ais_adapter.py:397: in _model_key
    json.dumps(self._build_ais_config(), sort_keys=True, separators=(",", ":")),
components/src/dynamo/planner/core/perf_model/ais_adapter.py:300: in _build_ais_config
    ForwardPassPerfModelConfig(
E   TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
== 80 failed, 6287 passed, 158 skipped, 2814 deselected in 2037.86s (0:33:57) ==

The same 80 failures as the amd64 sequential job, all from the same deployment_limits TypeError.

❌ dynamo-runtime / test / parallel cuda13.0, amd64: TypeError: unexpected keyword argument 'deployment_limits'

Job: dynamo-runtime / test / parallel cuda13.0, amd64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 124399138058 -R ai-dynamo/dynamo --log-failed

FAILED components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_predict_cli_cases[08-synthetic-throughput-planner] - AssertionError: aisimulate predict --stack dynamo ... failed with 1
  aisimulate predict failed: TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
FAILED components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[04-planner-preset-off-goodput] - AssertionError: aisimulate recommend --stack dynamo ... failed with 1
  File "/opt/dynamo/venv/lib/python3.12/site-packages/dynamo/planner/core/perf_model/ais_adapter.py", line 300, in _build_ais_config
    ForwardPassPerfModelConfig(
  TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
FAILED components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[06-disagg-router-planner-goodput] - AssertionError: aisimulate recommend --stack dynamo ... failed with 1
= 3 failed, 1378 passed, 33 skipped, 7925 deselected, 7 subtests passed in 1125.61s (0:18:45) =

3 test_unified_cli_dynamo.py cases fail because the aisimulate predict/recommend --stack dynamo subprocess exits with code 1. Its stderr shows the same deployment_limits TypeError from the planner's ais_adapter.py:300.

❌ dynamo-runtime / test / parallel cuda13.0, arm64: TypeError: unexpected keyword argument 'deployment_limits'

Job: dynamo-runtime / test / parallel cuda13.0, arm64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 124399138062 -R ai-dynamo/dynamo --log-failed

FAILED components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_predict_cli_cases[08-synthetic-throughput-planner] - AssertionError: aisimulate predict --stack dynamo ... failed with 1
  aisimulate predict failed: TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
FAILED components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[04-planner-preset-off-goodput] - AssertionError: aisimulate recommend --stack dynamo ... failed with 1
  File "/opt/dynamo/venv/lib/python3.12/site-packages/dynamo/planner/core/perf_model/ais_adapter.py", line 300, in _build_ais_config
    ForwardPassPerfModelConfig(
  TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'
FAILED components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[06-disagg-router-planner-goodput] - AssertionError: aisimulate recommend --stack dynamo ... failed with 1
= 3 failed, 1378 passed, 33 skipped, 7925 deselected, 7 subtests passed in 1620.44s (0:27:00) =

The same 3 aisimulate CLI cases as the amd64 parallel job, with the same deployment_limits TypeError in the subprocess stderr.

For agents
{
 "pr": 17743,
 "run_id": 41153898441,
 "run_attempt": 1,
 "head_sha": "39f0a5dff4f37d30543df94d656cf3ebd04a88dd",
 "failures": [
  {
   "job": "planner / CPU Test cpu, arm64",
   "job_id": 124398266810,
   "failed_step": "Run CPU-only tests (parallelized)",
   "signature": "TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'",
   "tests": [
    "components/src/dynamo/planner/tests/plugins/orchestrator/test_engine_adapter.py (44 tests)",
    "components/src/dynamo/planner/tests/unit/test_reconfiguration.py (18 tests)",
    "components/src/dynamo/planner/tests/unit/test_ais_perf_config.py (5 tests)",
    "components/src/dynamo/planner/tests/unit/test_signal_health.py (3 tests)",
    "components/src/dynamo/replay/tests/test_simulation_integration.py (2 tests)",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_predict_cli_cases[08-synthetic-throughput-planner]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[04-planner-preset-off-goodput]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[06-disagg-router-planner-goodput]"
   ],
   "log_cmd": "gh run view --job 124398266810 -R ai-dynamo/dynamo --log-failed"
  },
  {
   "job": "planner / CPU Test cpu, amd64",
   "job_id": 124398266854,
   "failed_step": "Run CPU-only tests (parallelized)",
   "signature": "TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'",
   "tests": [
    "components/src/dynamo/planner/tests/plugins/orchestrator/test_engine_adapter.py (44 tests)",
    "components/src/dynamo/planner/tests/unit/test_reconfiguration.py (18 tests)",
    "components/src/dynamo/planner/tests/unit/test_ais_perf_config.py (5 tests)",
    "components/src/dynamo/planner/tests/unit/test_signal_health.py (3 tests)",
    "components/src/dynamo/replay/tests/test_simulation_integration.py (2 tests)",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_predict_cli_cases[08-synthetic-throughput-planner]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[04-planner-preset-off-goodput]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[06-disagg-router-planner-goodput]"
   ],
   "log_cmd": "gh run view --job 124398266854 -R ai-dynamo/dynamo --log-failed"
  },
  {
   "job": "dynamo-runtime / test / sequential cuda13.0, amd64",
   "job_id": 124399138051,
   "failed_step": "Run CPU-only tests (parallelized)",
   "signature": "TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'",
   "tests": [
    "components/src/dynamo/planner/tests/plugins/orchestrator/test_engine_adapter.py (44 tests)",
    "components/src/dynamo/planner/tests/unit/test_reconfiguration.py (18 tests)",
    "components/src/dynamo/planner/tests/unit/test_ais_perf_config.py (5 tests)",
    "components/src/dynamo/planner/tests/unit/test_signal_health.py (3 tests)",
    "components/src/dynamo/replay/tests/test_simulation_integration.py (2 tests)",
    "components/src/dynamo/replay/tests/test_retune_replay.py"
   ],
   "log_cmd": "gh run view --job 124399138051 -R ai-dynamo/dynamo --log-failed"
  },
  {
   "job": "dynamo-runtime / test / sequential cuda13.0, arm64",
   "job_id": 124399138078,
   "failed_step": "Run CPU-only tests (parallelized)",
   "signature": "TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'",
   "tests": [
    "components/src/dynamo/planner/tests/plugins/orchestrator/test_engine_adapter.py (44 tests)",
    "components/src/dynamo/planner/tests/unit/test_reconfiguration.py (18 tests)",
    "components/src/dynamo/planner/tests/unit/test_ais_perf_config.py (5 tests)",
    "components/src/dynamo/planner/tests/unit/test_signal_health.py (3 tests)",
    "components/src/dynamo/replay/tests/test_simulation_integration.py (2 tests)",
    "components/src/dynamo/replay/tests/test_retune_replay.py"
   ],
   "log_cmd": "gh run view --job 124399138078 -R ai-dynamo/dynamo --log-failed"
  },
  {
   "job": "dynamo-runtime / test / parallel cuda13.0, amd64",
   "job_id": 124399138058,
   "failed_step": "Run CPU-only tests (parallelized)",
   "signature": "TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'",
   "tests": [
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_predict_cli_cases[08-synthetic-throughput-planner]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[04-planner-preset-off-goodput]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[06-disagg-router-planner-goodput]"
   ],
   "log_cmd": "gh run view --job 124399138058 -R ai-dynamo/dynamo --log-failed"
  },
  {
   "job": "dynamo-runtime / test / parallel cuda13.0, arm64",
   "job_id": 124399138062,
   "failed_step": "Run CPU-only tests (parallelized)",
   "signature": "TypeError: ForwardPassPerfModelConfig.__init__() got an unexpected keyword argument 'deployment_limits'",
   "tests": [
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_predict_cli_cases[08-synthetic-throughput-planner]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[04-planner-preset-off-goodput]",
    "components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[06-disagg-router-planner-goodput]"
   ],
   "log_cmd": "gh run view --job 124399138062 -R ai-dynamo/dynamo --log-failed"
  }
 ]
}

Posted automatically by Devin for run 41153898441. Updated on every full-CI run of this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feat planner size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant